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An  Overview  of  Classical  and  High  Resolution  Spectral 

Estimation 


Current  methods  of  spectral  estimation  can  be  broadly 
categorized  under  three  main  headings.  One  is  classical  power 
spectral  density  estimation  which  incorporates  estimation  of 
the  autocorrelation  function  through  lagged  products  and 
periodogram  analysis  and  its  variations.  The  second  is  power 
spectral  density  estimation  based  on  modelling.  This 

incorporates  maximum  entropy  analysis,  data  extension  using 
linear  prediction  and  spectral  estimation  using  ARMA  models. 
The  third  is  power  spectral  density  estimation  using  adaptive 
windows  which  incorporates  the  method  commonly  referred  to  as 
the  maximum  likelihood  (MLM)  method.  Classical  power  spectral 
density  estimation  typically  has  a  time  domain  and  a  frequency 
domain  interpretation.  The  time  domain  interpretation 

corresponds  to  estimating  the  autocorrelation  function  using 
lagged  products  of  the  given  data.  This  procedure  has  the 
characteristic  that  for  a  given  lag,  the  estimate  of  the 
correlation  is  unbiased  and  consistent  as  the  length  of  the 
data  record  increases,  but  no  matter  how  large  the  data 

record,  the  ends  of  the  lagged  products  remain  as  bad 
estimates.  Consequently,  the  lagged  products  are  typically 
windowed  to  retain  only  those  values  that  incorporated  a 

"sufficient"  number  of  points  in  the  estimate. 


In  the 
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frequency  domain,  classical  power  spectral  density  estimation 
is  interpreted  in  terms  of  the  periodogram  which  in  essence 
corresponds  to  the  magnitude  squared  of  the  Fourier  transform 
of  the  data  record.  The  procedure  is  particularly  convenient 
with  the  FFT  but  has  the  property  that  it  is  not  a  consistent 
estimate.  In  particular,  the  variance  of  the  estimate  is 
approximately  proportional  to  the  spectrum  being  estimated. 
There  are  various  ways  of  modifying  the  periodogram,  including 
periodogram  averaging,  smoothing  the  periodogram  or  a 
combination  of  windowing  the  data  and  smoothing  the 
periodogram. 

As  a  means  of  introducing  the  other  two  categories  of 
spectral  estimation  procedures,  it  is  useful  to  focus  on  what 
can  be  thought  of  as  two  philosophical  shortcomings  of 
classical  power  spectral  density  estimation.  In  terms  of  the 
interpretation  in  the  time  or  lag  domain,  classical  PSD 
estimation  provides  reasonable  estimates  of  the  correlation 
function  for  certain  lags,  but  implicitly  assumes  that  the 
correlation  function  is  zero  outside  the  interval  of  the 
window.  High  resolution  spectral  analysis  methods,  such  as 
MEM,  which  are  based  on  modelling,  can  be  thought  of  as 
providing  more  reasonable  ways  of  extending  the  data  or  the 
autocorrelation  function.  In  the  frequency  domain,  classical 
PSD  estimation  corresponds  to  smoothing  the  periodogram  with  a 
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fixed  window.  Thus,  the  same  smoothing  is  applied  whether 
there  are  narrow  lines  close  together  or  a  broad  spectral 
shape.  As  an  alternative,  use  of  an  adaptive  spectral 
smoothing  window  which  changes  its  shape  as  it  moves  across 
the  frequency  band  could  be  considered.  This  is  in  fact  an 
interpretation  of  the  maximum  likelihood  method  of  spectral 
analysis . 

The  most  prominent  procedure  under  the  category  of 
spectral  estimation  based  on  modelling  is  the  maximum  entropy 
method.  Theoretically  the  method  is  formulated  by  assuming 
that  the  true  autocorrelation  function  is  known  for  a  set  of 
lags  and  it  is  then  extended  in  such  a  way  that  the  entropy, 
defined  as  the  integral  of  the  log  spectral  estimate,  is 
maximized  subject  to  the  constraint  that  the  spectral  estimate 
matches  the  given  autocorrelation  value|.  It  can  be  shown 
that  this  procedure  is  equivalent  to  all-pole  modelling  of  the 
spectrum  and  in  turn,  all-pole  modelling  of  a  signal  can  be 
interpreted  as  a  procedure  of  data  extension  using  linear 
prediction.  These  two  interpretations  of  maximum  entropy  also 
suggests  several  generalizations,  one  being  the  more  general 
concept  of  a  parametric  model,  such  as  a  pole-zero  or  ARMA 
model.  The  second,  suggested  by  the  linear  prediction 
interpretation  of  maximum  entropy  suggests  extending  the  given 
data  on  the  basis  of  a  model  and  then  applying  classical 
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spectral  analysis  procedures  to  the  extended  data.  Other 
variations  on  the  maximum  entropy  method  derive  in  part  from 
the  fact  that  in  practice  it  is  the  data  record  which  is 
given,  not  the  autocorrelation  values,  and  consequently  the 
autocorrelation  values  must  be  estimated.  There  are  several 
methods  for  doing  this,  for  example,  the  correlation  method, 
the  covariance  method,  and  the  Burg  recursion. 

The  maximum  likelihood  method  of  spectral  estimation  is 
derived  from  a  different  viewpoint,  specifically  that  an 
attempt  be  made  to  smooth  the  underlying  spectrum  with  an 
adaptive  window  which  attempts  to  minimize  leakage  from 
neighboring  frequencies.  The  method  can  be  formulated  by 
imagining  filtering  of  the  data  with  an  FIR  filter  with  the 
constraint  that  the  filter  frequency  response  have  unity  gain 
at  the  frequency  at  which  the  spectrum  is  to  be  estimated. 
Subject  to  this  the  window  is  designed  to  have  minimum  output 
power  and  it  is  the  output  power  which  is  taken  as  the 
spectral  estimate.  When  the  resulting  analysis  was  carried 
through,  it  can  be  shown  that  the  resulting  spectral  estimate 
requires  the  autocorrelation  function  for  the  input  sequence 
out  to  a  number  of  lags  equal  to  the  length  of  the  FIR  filter. 
Since  the  autocorrelation  values  must  be  estimated  from  the 
given  data  record,  this  implies  that  the  underlying  FIR  filter 
in  the  formulation  of  the  method  must  be  much  shorter  than  the 
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length  of  the  data  record. 

There  are  a  wide  variety  of  tradeoffs  and  comparisons 
between  the  three  methods.  With  the  classical  method  the 
underlying  issue  is  principally  one  of  resolution.  The 
maximum  entropy  method  is  particularly  effective  at  separating 
narrow  lines  in  the  spectrum,  but  can  be  very  sensitive  to  the 
order  of  the  model  and  effects  such  as  peak  splitting  can 
occur.  In  addition,  the  maximum  entropy  method  tends  to 
follow  peaks  in  the  spectrum  more  closely  than  the  valleys. 
The  maximum  likelihood  method  provides  less  resolution  than 
the  maximum  entropy  method  as  is  clearly  demonstrated  through 
Burg's  resistor  averaging  formula  relating  the  two  methods. 
On  the  other  hand  the  procedure  is  more  robust  since  it  is  not 
as  susceptible  to  peak  splitting  as  the  MEM  method.  It  has 
also  been  empirically  demonstrated  that  for  the  MEM  method,  it 
is  the  area  under  a  peak  in  the  spectral  estimate  that  is  an 
indication  of  the  signal  power  at  that  frequency,  whereas  with 
the  MLM  method  it  is  the  amplitude  of  the  peak.  For  classical 
spectral  analysis,  both  the  peak  amplitude  and  the  area  under 
the  peak  tend  to  be  proportional  to  the  signal  power. 

In  summary,  all  three  methods  are  derived  on  the  basis  of 
different  assumptions  and  there  are  resulting  character istics 
and  potential  hazards  particular  to  each  of  the  methods. 
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